Integration of Geometric and Conceptual Reasoning for Interacting with Virtual Environments

This paper describes the knowledge processing in the CODY Virtual Constructor, an operational system enabling the interactive assembly of complex aggregates in a virtual environment. Two forms of reasoners axe used: a geometric reasoner that infers spatial properties of scene objects and a conceptual reasoner that keeps track of the evolving aggregate’s assembly structure. The combination of the two reasoners enables the system both to simulate assembly processes in the virtual environment and to understand natural language instructions. By maximizing the mutual exchange of information between the reasoners, additional knowledge can be inferred that not only improves understanding of language instructions but also increases efficiency of inferencing.